ASME's Mechanical Engine…ing Toolkit 1997 December/ASME's Mechanical Engineering Toolkit 1997 December.iso/win_eng/atre27.exe/ATREE_27/BEGINNER.TXT

Text File | 1992-08-01 | 12KB | 226 lines

    Beginner's introduction to the atree release 2.7
    adaptive logic network (ALN) simulation software.

Welcome to logical neurocomputing and the atree release 2.7 software!
This package will help you experiment with learning networks in ways
that will instruct, and perhaps surprise you.  The package might even
be adequate for building some industrial applications, but it is
really intended for the researcher and experimenter.

Goals of neurocomputing.

The field of neurocomputing dates back to the 1940s, at least, and has
many branches.  One particularly important division is into two
groups: those whose goal it is to explain how animals' brains and
nervous systems really work, and those who pursue other goals.  The
latter group direct their efforts to solving artificial intelligence
problems, like the control of robot motion or the exploitation of
information in images or sounds, using whatever computing tools seem
appropriate, whether biologically relevant or not.

It is the latter group that atree is meant for, those who want to
engineer "intelligent" systems.  The word "intelligent" is placed in
quotes here, simply because we have very limited expectations of what
computers can "understand" today.  It would be exciting if ALNs held
the secret of building "advanced neural network processors", as
referred to in the recent film "Terminator II", but, alas, the most we
can hope for is that logical neurocomputing may provide one small
component of such future systems.  This might analogous to knowing how
to fabricate transistors on a chip, but not knowing how to put them
together to get a calculator.

Since people are very sensitive to "hype" in the area of neural
networks, it is desirable not to promise anything until you have done
it.  Fortunately, there are many demonstrations of practical
application of neurocomputing today, so the field can exist without
the hype of unrealistic promises.

How neurocomputing is applied today.

If you have just heard the term neural networks, but have never used
one, then the following description will give you an idea what to
expect from using this software.

Many problems cannot be completely solved by mathematical techniques,
or by standard software development techniques, even though there may
be a wealth of empirical data available.  Just having this data is
often a good step towards solving the problem, and neural networks are
an important tool in linking the data to the solution.

For example, consider a medical application where some measurement
data related to symptoms, treatments and history are given for each
person in a sample, together with a value indicating whether a certain
disease was found present or not (after costly tests).  Suppose we
want to find a simple relationship between the given data and the
presence of the disease, which could lead to an inexpensive method for
screening for the disease, or to an understanding of which factors are
important for disease prevention.  If there are many factors, with
complex interactions among them, the usual "linear" statistical
techniques may be inappropriate.  In that case, one way of analysing
data is by the use of neural networks, which can, in principle, find
simple relationships by means of an adaptive learning procedure in
very general circumstances.  Since the method uses only empirical
data, very little human intervention may be required to obtain an
answer, making the approach very easy to try out.  The MOSQUITO
example included in the release, is an illustration.

Beyond applications in data analysis, such as the above, ANS are being
used in an increasing number of applications where high-speed
computation of functions is important.  For example, to correct an
industrial robot's positioning to take into account the mass of an
object it is holding would normally require time-consuming numerical
calculations which are impossible to do in a real-time situation
during use of the robot.  An ANS can learn the necessary corrections
based on trial motions, and can perform the corrections in real time.
It is not necessary to use all possible masses and motions during
training, since ANS have the capacity to extrapolate smoothly to cases
not presented in training.

Extremely high speed response is needed in electronic systems, when
parameters have to be adjusted, as in automatic equalizers for
communication links.  Other applications are in pattern recognition,
sensor fusion, and sonar signal interpretation.

Can ALNs compute only logic functions?

No.  Although the networks are constructed using logical operations,
it is important to realize that they can also be applied to functions
of real values or integers by using appropriate encodings of reals or
integers into logical vectors.  The results of logical computations
are then decoded to obtain the real or integer results.  This feature
has been included and improved in release 2.5.

Comparison between ANS in general and ALNs.

The most prevalent approach to neural network learning is the
backpropagation algorithm and its variants.  Backpropagation depends
on fast multiplications and additions, as well as table lookup of
functions called sigmoids.  Special processors have been built to do
ANS computations at high speed.  Some systems resort to analog chips
for greater speed.

The ALN is a type of ANS that has been developed at the University of
Alberta, following earlier work at Bell Telephone Laboratories and the
Universite de Montreal.  It uses only simple logical functions such as
AND, OR, and NOT.  Execution hardware is just an ordinary
combinational logic circuit, a digital logic circuit without cycles or
delay units.  A learning system of this kind we call an adaptive logic
network (ALN).  The atree software is a simulator for ALNs.

The ALN approach is an extremely simplified version of the
backprogagation approach -- with some enhanced heuristics for
learning.  Though simple, it can replace backpropagation in many
situations.  Where we expect ALNs to be of particular use is in two
situations: when one needs to solve some pattern recognition problem
at very high speed, or when one is restricted to using a computer with
very limited processing power (for example a personal computer with no
math co-processor).  We think all PC users will be pleased with the
potentially faster learning and execution ALNs offer them.

Comparisons with the description of a commercial processor using
standard ANS suggested that hardware based on ALNs could evaluate
functions several orders of magnitude faster, and would be in the
trillion connections per second range.  (A connection is a
multiply-add operation, and in a NAND (NOT_AND) gate, this is a
special case which can be done at very high speed, e. g. 2 nanoseconds
or less.  The computation time for a tree is proportional to its
depth, not its size.)

Another advantage of the logic networks is that most of a computation
can often be left out.  For example if a logical zero is input to an
AND-node in a logical tree, then the output of the node can be
computed without computing the other input, or even knowing the inputs
which give rise to it.  This produces no speedup in a completely
parallel system, however in systems running on ordinary processors, or
in systems which re-use any ANS special hardware sequentially (the
usual case), it is of critical importance for speed.  Systems which
combine special hardware parallelism with the possibility of leaving
out unnecessary computations are using "parsimonious parallelism".  A
small amount of such parsimony applies to ordinary ANS, but in a
logical network, the speedup produced can amount to many orders of
magnitude, which confers a great advantage on this approach vis-a-vis
the usual one.  The "fast-tree" feature, new to release 2.5, turns
pattern recognition problems into traversal of a binary decision tree.

The backpropagation technique for training standard ANS is quite slow.
The simulation of ALN learning should prove fast enough in many
situations, and we hope will prove satisfactory even on personal
computers.  Parsimony plays an important role in learning too.

There is a hardware architecture for training adaptive logic networks
which would make on-line learning quite feasible.  A high-speed
adaptive processor will soon, we hope, eliminate long waits for
convergence of training for all but the most difficult tasks.  It will
process patterns at a rate of 25 million per second!  All the learning
tasks we have tried so far would finish in a small fraction of a
second.  This possibility will undoubtedly open up completely new
fields of application for ALNs.

Acknowledgment.

The production of this package was done mainly by Andrew Dwelly, Rolf
Manderscheid and Monroe Thomas, to whom I am very grateful for their
dedicated work.  They only sought other employment when faced with the
alternative of starvation.  Important contributions have been made by
many others, including Scott Reynolds, Dekang Lin, and Jiandong Liang.
Financial support came from the Natural Sciences and Engineering
Research Council of Canada.  It has supported this type of work on and
off for over twenty three years, which I acknowledge with thanks.

Future developments.

What we have been able to incorporate into release 2.5 just scratches
the surface of the possibilities for ALNs.  Unfortunately, all my
current research money has been spent on the above high-quality
programming talent, and those people have now moved on to other jobs.
Rolf Manderscheid and Monroe Thomas have kindly agreed to help with
getting any remaining bugs out of release 2.5 for the Unix and PC
versions respectively however.

Release 2.7 will be the last research-oriented one.  Because the
difficulties of getting adequate support for research into ALNs in
Canada appear insurmountable, it is time for this technology to leave
the research nest and fly on its own.  I am convinced it is a winner,
and that some of you will take up the challenge of making it work for
you.  If someone succeeds in developing a commercial prototype based
on the software, I expect commercial licensing of that derived program
would be possible, but details would have to be discussed with the
authors and the lawyers at the U. of A.  If someone wanted to develop
a more advanced system, which some of our research has shown to be
possible, that is a question of providing funding.  The development of
the high-speed hardware adaptive processor, the "jewel" of the entire
ALN technology, is up for grabs, and would be along the more advanced
lines alluded to above.  If there is some government agency or
commercial organization that would be interested, something could be
done there.

Until the advent of commercial software and harware, I extend my best
wishes for success in using the atree release 2.7 package!  Please try
a few experiments by just changing the given sample programs first.
After you gain confidence, I hope you will try some really exciting
projects.  Please let me know about your experiences, so future
adaptive logic networks can be better.

Bill Armstrong


Address:

    Professor William W. Armstrong
    Department of Computing Science
    University of Alberta
    Edmonton, Alberta
    Canada
    T6G 2H1
    Tel. [phone]
    FAX: [phone]
    email: [e-mail]

Note: If you can't obtain atree release 2.7 from a friend or through
ftp from menaik.cs.ualberta.ca [129.128.4.241] in pub/atre27.exe, you
can order it by mail.  The programs for both Unix (TM AT&T) and PC
versions and documentation, including previously published papers and
some unpublished ones, will be mailed to you upon payment of $150
(Canadian), made payable to the University of Alberta.